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Abstract 

In this note, we describe a probabilistic attack on public key cryptosystems based on the word/conjugacy 
problems for finitely presented groups of the type proposed recently by Anshel, Anshel and Goldfeld. In such 
a scheme, one makes use of the property that in the given group the word problem has a polynomial time 
solution, while the conjugacy problem has no known polynomial solution. An example is the braid group 
from topology in which the word problem is solvable in polynomial time while the only known solutions to 
the conjugacy problem are exponential. The attack in this paper is based on having a canonical representative 
of each string relative to which a length function may be computed. Hence the term length attack. Such 
canonical representatives are known to exist for the braid group. 



1. Introduction 

Recently, a novel approach to public key encryption based on the algorithmic difficulty of solving the word and 
conjugacy problems for finitely presented groups has been proposed in Pll2l l20l 1211 . The method is based on 
having a canonical minimal length form for words in a given finitely presented group, which can be computed 
rather rapidly, and in which there is no corresponding fast solution for the conjugacy problem. A key example 
is the braid group. In this note, we will indicate a possible probabilistic attack on such a system, using the 
length function on the set of conjugates defining the public key. Note that since each word has a canonical 
representative, the length function is well-defined and for the braid group can be computed in polynomial time 
in the word length according to the results in [6 1. The attack may be relevant to more general types of string 
rewriting cryptosystems, and so we give some of the relevant background. Thus this note will also have a 
tutorial flavor. 

The contents of this paper are as follows. In Section 2, we make some general remarks are rewriting 
systems, and the notion of "length" of a word. In Section 3, we define the Artin and Coxeter groups. In Section 
4, we discuss the classical word and conjugacy problems for finitely presented groups. In Section 5, the braid 
cryptosystem of 1 1 1 is described. In Section 6, we give the length attack for possibly compromising such a 
cryptosystem, and finally in Section 7 we draw some general conclusions, and directions for further research 
for group rewriting based encryption systems. 
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2. Background on Monoid and Group Based Rewriting Systems 

In this section, we review some of the relevant concepts from group theory for rewriting based encryption. We 
work in this section over a monoid, but similar remarks hold for group based rewriting systems as well. 

Let k be an arbitrary field, and S — {a\, . . . , a n } a finite set. Let S* be the finite monoid generated by S, 
that is, 

S* = {a 1 , a %n , 

L tr(l) (r(n)J 

Elements of S* are called words. We then define the free algebra generated by S to be 

A = k[S*] = k<S>={J2 kn...iA(i) ■ ••<"(„)>' 

<t6E„ 

where £„ denotes the symmetric group on n letters. 

We are now ready to define precisely the key notion of rewriting system. Let R C S* x S*. We call R the 
set of replacement rules. Many times the pair (u, v) G R is denoted by u — > v. The idea is that when the word 
u appears inside a larger word, we replace it with v. More precisely, for any x, y G S*, we write 

xuy — » xvy, 

and say that the word xuy has been re-written or reduced to xuy. x is irreducible or normal if it cannot be 
rewritten. 

We will still need a few more concepts. We say that the rewriting system (S, T) is terminating if there is 
no infinite chain x — > x\ — > X2 — > • • ■ of re-writings. We then say that the partial ordering x > y defined by 
x — > • • • — > y is well-founded. R is confluent if a word x which can be re-written in two different ways yi and 
?/2, the re-writings y\ and ?/2 can be re-written to a common word z. 

Note that if R is terminating, confluence means that there exists a unique irreducible word, x re d representing 
each element of the monoid presented by the re-writing system. Such a system is called complete. Given a 
word x G S*, we define the length of x or £(x), to be the number of generators in x re d- 

Remark: 

In the case of groups, the basic outline just given is valid. A key example of a group in which one can assign a 
length function is the braid group via the results in 1 6 1 . This is the basis of the cryptosy stem proposed in 1 1 1 . 

3. Artin and Coxeter Groups 

In this section, we review some of the pertinent background on Artin and Coxeter groups. An excellence 
reference for this material in J5|, especially for the braid groups. 

Let G be a group. For a, b G G we define 

< ab > q :— aba . . . , product with q factors. 

For example, 

< ab > 3 :— aba, < ab > 4 := abab, < ba > 5 :— babab. 

An Artin group is a group G which admits a set of generators {a^i^i with / a totally ordered index set, and 
with relations of the form 

< didj > m 'J=< a^i > m ^, 
for any i,j G I and with my non-negative integers. The matrix M := [my]; jS / is called the Coxeter matrix. 

2 



Length-Based Attacks for Certain Group Based Encryption Rewriting Systems 



The braid group, B n , is defined by taking the indexing set I := {1, . . . , n}, and 

rriij = 2 for \i — j \ > 1 , 
rn M+ i = mi+i,j = 3. 

Thus the braid group B n is a special case of an Artin group defined by the generators a\, . . . , o~ n , with the 
relations 

OiOj = o-jcri \i - j\ > 1, i,j G I, 
o~iO~i+i<Ti = ai + iai(7i + i . 

Given an Artin group G with Coxeter matrix M := [m^Ji je / the associated Coxeter group is defined by 
adding the relations af = 1, for t 6 J. One can easily show them that a Coxeter group is equivalently defined 
by the relations 

{aia,j) mi] — 1, i, j G I, with ma = 2. 

Artin groups and their associated Coxeter groups have some nice properties which could make them quite 
useful in potential rewriting based systems as we will now see. 

4. Word and Conjugacy Problems for Finitely Presented Groups 

Let 

G =< sx,s 2 , ■■■ ,s n : r%,--- ,r fe > 

be a finitely presented group. Let U be the free monoid generated by Sj and s^ 1 . Then the word problem is 
given two strings (words), u,v G U, decide if u — v in G. The conjugacy problem is to decide if there exists 
a G G such that u = ava^ 1 , i.e., u and v are conjugates. 

It is well-known that both these problems are algorithmically unsolvable for general finitely presented 
groups. However, for some very important groups they are solvable, e.g., for Artin groups with finite Coxeter 
groups. In fact, Brieskorn and Saito |8| give an explicit solution to the word and conjugacy problems for this 
class of groups. Their algorithm runs in exponential time however. See also 1 13 14 1 and the references therein 
for some recent results on the word and conjugacy problems for Coxeter groups. 

In some recent work, Birman-Ko-Lee |6| show that for the braid group, the word problem is solvable in 
polynomial time (in fact, it is quadratic in the word length). Given the results just described, it has been 
conjectured that the techniques of |6| are extendable to Artin groups with finite Coxeter groups. For another 
solution to this problem see [ 1 1 1. 

At this point, there is no known polynomial time algorithm known for the conjugacy problem, as originally 
posed by Artin [3 1, for the braid group with n > 6 strands; see |6|. It seems that it is the possible complexity of 
this form of the conjugacy problem which is the basis of the claim of security made by the authors of the braid 
cryptosystem in 1 1 1. (The original conjugacy problem posed by Artin is a decision problem. Given x, y G B n , 
is there an a such that x = a~ 1 xal In the proposed cryptosystems, the public and private keys are known to be 
conjugates, so these systems are not based on such a decision problem. ) 

For the braid group itself, little work has been accomplished on the lower and average bounds of the 
conjugacy search problem for known conjugates (as in 1211 ) or a system of known conjugates (as in Q). 
There are no proofs that the conjugacy problem is hard all the time. The motivation to do any of this work has 
only occurred recently because these cryptosystems have been proposed. Some of this work includes a brief 
look at the probabilities of colored Burau representation 1171 . and other work attempting to demonstrate the 
average complexity of the conjugacy problem [23 24 1 using a set measurement techniques for infinite groups 
1 7 1. Other work has begun on calculating the normalizer set to solve the conjugacy problem 1 15 1 (but this does 
not help solve the crypto-problem because it assumes a known conjugator exists.) 
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It is important to note that there are some important linear representations of the braid group namely, 
the Burau, the colored Burau and the Lawrence-Krammer. In [2|, it is suggested that the colored Burau 
representation made be used to quickly solve the word problem. The Burau representation was originally 
formulated to prove that the braid group was linear, but it now is known to have a non-trivial kernel and as 
such, cannot be used to solve the general conjugacy problem. Finally, using a more general representation due 
to Lawrence-Krammer, it has been proven that the braid group is indeed linear |4 1. This allows linear algebraic 
methods to be used now in studying the word and conjugacy problems, and possibly could lead to yet another 
attack on braid cryptosystems. 

Finally, note that if one can find a unique irreducible word from which one can derive a length function, then 
one can give a natural distance between words in a given group G. Indeed, let a, ft, 7 denote words relative to 
a finite presentation of the group G. Let £ denote the length function which we assume exists. Then we define 
the distance d between the words a, (3 as 

d G (a,f3) :=l{ap- 1 ). 

It is trivial to check that da is a distance function function between words. See also 1 13 1. We will see that this 
is the case for the braid group via the results of Birman-Ko-Lee 1 6 1 . 



5. Braid Cryptosystem 

In some very interesting recent work, Anshel et al. [1 2 1 propose a new twist to rewriting systems for public 
key encryption. We will first state their approach over a general group. We should first note however that the 
use of the word and conjugacy problems for public -key cryptosystems is not new. An early reference is [26 1. 

The general idea is as follows: Alice (A) and Bob (B) have as their public keys subgroups of a given group 

G, 

Sa =< si, ■ ■ ■ , s n >, <ti,...t m >. 

A chooses a secret element a G Sa and B chooses a secret element b G Sb- A transmits the set of elements 
a~ 1 tia 1 . . . , a _1 i m a and B transmits the set of elements & -1 si&, . . . , b~ 1 s n b. (The elements are rewritten is 
some fashion before transmission.) 

Now suppose that 

a — S er(l) ' ' ' S a(n)- 

Then note that 

= b- 1 s ll n ,bb- 1 s l *b---b- 1 s l ", ,b 
= (&- 1 Mi)&) <1 ---(& _1 M») & ) <n - 

(The conjugate of the product of two elements is the product of the conjugates.) Thus A can compute 6^ 1 a6, 
and similarly B can compute a~ 1 ba. The common key then is 

a~ 1 b~ 1 ab — [a, b], 

the commutator of the two secret elements. 

Note that since the two users have the common key written in different forms, in order to extract the 
message, it must be reduced to an identical group element. For the braid group, this can be accomplished by 
reducing the commutator to the Birman-Ko-Lee canonical form [6 1, colored Burau 1 2 1 or Dehornoy 1 1 1. 

The braid group is particularly attractive for this protocol since one has a quadratic time solution for the 
word problem, and the only known solution to the conjugacy problem is exponential. 
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Remark: 

The key properties that underlie this cryptosystem are having a group in which the word problem is easy to 
solve (and in fact each word has a canonical form) and in which the conjugacy problem is difficult (at least via 
known techniques). The canonical form is important as well since it allows a simple method for the extraction 
of the common key. 



5.1. Another Braid Cryptosystem 

Another possible cryptosystem based on the word and conjugacy problems in the braid group has been proposed 
in 1201 1211 . In this case, the authors propose the following scheme: Consider the braid group B n+m on 
n + m braids. One considers two subgroups: LB„ generated by <j\, . . . , <t„_i and RB m generated by 
OYi+i, . . . cr n + m _i. Note that given a £ LB n and b 6 RB m , ab = ba. This is essential for their scheme. 

The protocol for creating a common key then works as follows: The public key is a pair of integers (n, m), 
and braid x G B n+m . Alice choose a secret element a G LB n and sends axa^ 1 to Bob. Bob chooses a 
secret element b G RB m , and sends bxb^ 1 to Alice. Alice can compute a(bxb~ 1 )a~ 1 and Bob can compute 
b(axa~ 1 )b^ 1 . Since a and b commute this is the common key. Being able to solve the Generalized Conjugacy 
Problem would be enough to break this system. It is not known if the converse is true. 

Remark: 

It is an interesting open question to see if the length attack proposed below may be suitably modified to be 
relevant to the protocol in [20|. It may be also be of interest to consider some the strong convergent game- 
theoretic techniques in HI 31 1141 to study this protocol as well as that in (Q. 

6. The Length Attack 

In this section, we describe the length attack on word/conjugacy based encryption systems of the type 
proposed in in which one can associate a canonical representative, and therefore a length function I of 
the type described above. For concreteness, we focus on the braid group here which has a canonical length 
function as noted above. We should note that the arguments of this section are speculative, and certainly not 
mathematically rigorous. 

Research on the length of random words has been done in the mathematical physics community where braid 
group has been valuable in studying certain physical phenomena I9l ll2ll25l . Recall that a symmetric random 
walk on a free group T n with n generators is a cross product of a nonsymmetric N-step random walk on a 
half-line Z + and a layer over N G Z + giving a set of all words of length N with the uniform distribution (see 
1251 for the details). The transition probabilities in a base are: 




N +1 with the probability 
— 1 with the probability ^ 



It is easy to show then that the expectation of a word's length after N steps is 



and hence the drift is 

n - 1 
n 

In 1125 1 , the authors show that while the statistical properties of random walks (Markov chains) on locally 
free and braid groups are not the same as uniform statistics on these groups, nevertheless the statistical 
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characteristics stabilize as the number of generators n grows. ^From this fact, for large n (see [25 1, Theorem 
11) given two generic words x, y <E B n , the length of xy will be approximately i(x) + £(y). (The genericity is 
important here. For example, if x = y^ 1 then £(xy) = 0.) This does give some statistical backing to the length 
attack which we are about to formulate. 

Given x 6 B n , we say that y is a reducing with respect to x (or a reducing element if x is understood), if 

e(y- x xy) < £{x). 

The remainder of this discussion will be a way of using substantial reducing strings in a length attack, and 
calculating an upper bound for the actual difficulty of this attack. It is important to emphasize that the ability 
of removing large reducing elements is not a general solution to the braid conjugacy problem. It is a specific 
attack on word/conjugates encryption systems of the type defined 1 1 1. Indeed, for such cryptosystems one has 
the some key information about the secret elements, namely, the factors are known and their number bounded. 

Let 

a e Sa =< si, . . ., s„ >, 

be the secret element. Recall that in the above protocol, a~ 1 t r a and t r (r = 1, . . . m) are publicly given. We 
also assume that the factors Sj have lengths large relative to a. For given r, set 

u r = a~ 1 t r a. 

Then the idea is to compute 

repeatedly. If Sj is a reducing string with respect to u r , then one has found a correct factor of a with a certain 
probability which will depend on the lengths £(s-t) for i = 1, . . . , n. The key is that the canonical lengths £(si) 
should be large. In this case, there is the greatest probability of a reducing string being formed which can be 
used to glean information about a. 

We can estimate the workload in carrying out such a procedure. Without loss of generality we can assume 
that a is made up of n distinct factors combined in d ways. If the length of the Si is large, then one join a 
small number of these factors together to create a substantial reducing string. If we include the inverses of 
the generators, we should consider 2n factors. Let us call the number of factors necessary to make a reducing 
string k. Thus we can create (2n) k reducing factors to try. 

By trying all reducing elements, a pattern that there are certain factors which annihilate better than others 
should be observed. One can do this on a single public conjugate in (2n) k operations. This pattern can be 
significantly reinforced by repeating this n times on each public conjugate at r a _1 . Combining all the steps 
above brings us to n(2n) operations. 

Relative to the lengths £(si) of the generators Si (and the specific group chosen), we conjecture that in 
a number of cases this will be sufficiently reliable to removing a given Si, so that backtracking will not be 
necessary. We can now do this dn times bringing the total to dn(2n) k operations. 

This is polynomial to the number of different factors, and linear to the number of factors in the public keys. 
This is the basis of the length attack. 

Another demonstration of this idea is trivial. If one sets k = d then the first of the dn passes will solve 
the system in the expected exponential time 2n k steps. This is simply an enumeration of all possible values of 
a. If one sets k = 1 then, if individual factors are not significant, this attack will not work. If there is a value 
of k < d that works, this attack significantly reduces the strength of the result. Once this attack is valid, then 
lengthening the private key only linearly increases the time to solution. 

Depending on the values chosen for the cryptosystem in (fl, k may need to be longer than the actual word 
a, as has been suggested in 1 2 1 1 . Yet another potential problem is that if the factors are simple, other attacks 
such as those proposed in 1 19 1 may be effective. 

1 This attack was known to the authors before that paper was written 



6 



Length-Based Attacks for Certain Group Based Encryption Rewriting Systems 



In some sense, the length attack is reminiscent of the "smoothness" attack for the Diffie-Hellman public key 
exchange system based on the discrete logarithm \ 22\. In this case, the protocol may be vulnerable when all of 
the prime factors of q — 1 (where the base field for the discrete logarithm has q elements) are small. (Such a 
number is called smooth.) 



7. Conclusions 



We have made a computation which indicates that a length attack on a conjugacy/word problem cryptosystem 
of the type defined in 1 1 1 has difficulty bounded above by dn(2n) . Given this conjecture, the only exponential 
aspect is the number of factors necessary to form a reliable reducing string. To make this secure, k needs to be 
100 or larger. 

In addition, as described, this attack does not use many tricks that one can use in order to speed up this 
length algorithm by several orders of magnitude. These include randomized and/or genetic algorithms which 
lead to more probabilistic solutions. 

The bottom line is that the length attack forces one to take generators of not too long canonical length. 
Dorian Goldfeld reports that experimental evidence suggests that if each of the generators sj., . . . ,s„ is of 
length < 10 in the Artin generators, then this may foil the attack. All of this still must be tested. 

Finally, it is important to note that this attack does not solve the general conjugacy problem for the braid 
group. Indeed, in this case the factors of a are known and bounded. In the general conjugacy problem, the 
number of possible factors of a is infinite. Consequently, the the conjugacy problem seems to be much harder 
and not amenable to this technique. The key exchange of the type proposed in 1 1 1 requires the factors be known 
and communicated, and give the attacker far more information than is known to the general conjugacy problem. 
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